Unsupervised Feature Selection Research Articles

Precise feature selection via non-convex regularized graph embedding and self-representation for unsupervised learning

In graph embedding learning with unsupervised feature selection fields, similarity matrices are usually obtained only from the initial noise-laden samples, and the probable association between various features is often ignored. To tackle the issues described above, this article proposes an effective unsupervised group feature selection approach via non-convex regularized graph embedding and self-representation (NLGMS). NLGMS integrates the similarity matrix and feature selection as a joint system on the basis of graph. It combines self-representation learning into the system to preserve the global structure and learns the nonconvex projection matrix to direct the procedure of feature selection. Specifically, first, NLGMS presents a unified framework for global structure learning, local structure learning, and feature selection process so the data’s intrinsic structure is excellently captured and learned adaptively. Second, nonconvex ℓ2,0-norm is enforced upon the projection matrix to perform the process of feature selection, ensuring that the optimal feature subset can be picked precisely. Nonconvex ℓ2,0-norm constraint considers the probable association between various features so that it can deliver optimal performance by considering the feature subset as an integral whole, which other convex regularized constraints often fail to achieve. Finally, the introduction to self-representation learning can facilitate the reconstruction of samples in low-rank subspaces, strengthening the model’s robustness. To tackle this challenging nonconvex model, an innovative alternative optimization technique is exploited. In addition, to validate the superiority of NLGMS, exhaustive experiments are performed on one synthetic dataset, six benchmark datasets, and one fish image dataset. The source code is available at: https://github.com/hrbai/NLGMS.

Dual-level feature assessment for unsupervised multi-view feature selection with latent space learning

Recently, numerous unsupervised multi-view feature selection methods have been presented. However, these methods assess the significance of data features in each view individually or jointly evaluate the significance of data features across multiple views by concatenating data features from all data views. So, they disregard either inter-view or intra-view feature connections of data. Moreover, they ignore the complementary information from multiple insufficient views, each of which only captures partial information of multi-view data and is insufficient to characterize data distribution. To address these problems, this paper proposes a model named Dual-level Feature Assessment for Unsupervised Multi-view Feature Selection with Latent Space Learning (DFA-LSL). It first investigates the underlying complementary information from multiple views in a latent space while simultaneously learning latent representations for data. Compared with the original data representations, the latent representations contain the entire information of multi-view data. Then, it explores connections and distinguishing ability of features at two levels, namely the inter-view and intra-view levels, by jointly assessing the significance of features in the latent space and individually assessing the significance of features in each view concurrently. Following that, an effective optimization algorithm is presented. Extensive experiments demonstrate that the proposed work surpasses several state-of-the-art models.

Self-paced regularized adaptive multi-view unsupervised feature selection

Multi-view unsupervised feature selection (MUFS) is an efficient approach for dimensional reduction of heterogeneous data. However, existing MUFS approaches mostly assign the samples the same weight, thus the diversity of samples is not utilized efficiently. Additionally, due to the presence of various regularizations, the resulting MUFS problems are often non-convex, making it difficult to find the optimal solutions. To address this issue, a novel MUFS method named Self-paced Regularized Adaptive Multi-view Unsupervised Feature Selection (SPAMUFS) is proposed. Specifically, the proposed approach firstly trains the MUFS model with simple samples, and gradually learns complex samples by using self-paced regularizer. l2,p-norm (0<p≤1) is employed to measure the learning error and as the sparse regularization to accommodate various sparsity requirements across different datasets. Moreover, hypergraph Laplacian matrices are constructed for each view to better preserve the local manifold structure and encode high-order relationships within the data space. They are adaptively assigned weights to learn the underlying correlated and diverse information among different views. An iterative optimization algorithm is proposed to solve SPAMUFS and the convergence and computational complexity are also analyzed. The effectiveness of SPAMUFS is substantiated by comparing with eight state-of-the-art algorithms on nine public multi-view datasets.

Unsupervised text feature selection by binary fire hawk optimizer for text clustering

Unsupervised text feature selection by binary fire hawk optimizer for text clustering

DELVE: feature selection for preserving biological trajectories in single-cell data

Single-cell technologies can measure the expression of thousands of molecular features in individual cells undergoing dynamic biological processes. While examining cells along a computationally-ordered pseudotime trajectory can reveal how changes in gene or protein expression impact cell fate, identifying such dynamic features is challenging due to the inherent noise in single-cell data. Here, we present DELVE, an unsupervised feature selection method for identifying a representative subset of molecular features which robustly recapitulate cellular trajectories. In contrast to previous work, DELVE uses a bottom-up approach to mitigate the effects of confounding sources of variation, and instead models cell states from dynamic gene or protein modules based on core regulatory complexes. Using simulations, single-cell RNA sequencing, and iterative immunofluorescence imaging data in the context of cell cycle and cellular differentiation, we demonstrate how DELVE selects features that better define cell-types and cell-type transitions. DELVE is available as an open-source python package: https://github.com/jranek/delve.

Boosted unsupervised feature selection for tumor gene expression profiles

AbstractIn an unsupervised scenario, it is challenging but essential to eliminate noise and redundant features for tumour gene expression profiles. However, the current unsupervised feature selection methods treat all samples equally, which tend to learn discriminative features from simple samples. Besides, it is difficult for these methods to guide the feature selection process by multiple feature subspaces comprehensively. In order to address these problems, a boosted unsupervised feature selection method (BoostUFS) is proposed for tumour gene expression profiles. Specifically, the authors design a boosting scheme to sequentially learn multiple compressed feature subspaces by focusing on ambiguous samples. The uncertainty of samples and the confidence of feature subspaces can be evaluated adaptively by minimising the overall loss of feature subspaces learning. Furthermore, we provide a consensus objective function with L2,1‐norm regularisation to combine these weighted feature subspaces and select discriminative features. Extensive experiments on several real‐world datasets of tumour gene expression profiles are carried out to demonstrate the effectiveness of the proposed method.

Graph embedding orthogonal decomposition: A synchronous feature selection technique based on collaborative particle swarm optimization

In unsupervised feature selection, the clustering label matrix has the ability to distinguish between projection clusters. However, the latent geometric structure of the clustering labels is often ignored. In addition, the optimal sub-feature selection performance of feature selection techniques relies greatly on the choice of balanced parameters, and the selection range of most technical parameters is limited and fixed. To solve the above-mentioned problems, this paper proposes a synchronous feature selection technique based on graph-embedded cluster label orthogonal decomposition and collaborative particle swarm optimization (GOD-cPSO). First, GOD-cPSO extends the feature selection framework of clustering label orthogonal decomposition by graph embedding to retain the latent geometric structure of clustering labels, thus maintaining the correlation between clustered sample labels. Then, the l2,1-2-norm with strong global convergence is extended to the graph embedding clustering label orthogonal decomposition framework. By imposing this non-convex constraint, GOD-cPSO can achieve low-dimensional sparse and low-redundant sub-features. In addition, the local structure preserving of low-dimensional manifolds is integrated into the graph-embedded clustering label orthogonal decomposition framework to obtain good cluster separation and effectively maintain the latent local structure of the data. Finally, to ensure the adaptive parameter selection over a large range, GOD-cPSO synchronously guides the graph-embedding clustering labeling orthogonal decomposition framework for feature selection through collaborative particle swarm optimization. GOD-cPSO has synchronous parameter optimization and feature selection and picks parameters in a larger range. Comprehensive numerical experiments are performed on nine datasets to test the validity of the GOD-cPSO. The experimental results demonstrate that the sub-features selected by the GOD-cPSO have stronger discriminative power and are superior to other techniques in the clustering assignments.

Selective Deep Autoencoder for Unsupervised Feature Selection

In light of the advances in big data, high-dimensional datasets are often encountered. Incorporating them into data-driven models can enhance performance; however, this comes at the cost of high computation and the risk of overfitting, particularly due to abundant redundant features. Identifying an informative subset of the features helps in reducing the dimensionality and enhancing model interpretability. In this paper, we propose a novel framework for unsupervised feature selection, called Selective Deep Auto-Encoder (SDAE). It aims to reduce the number of features used in unlabeled datasets without compromising the quality of information obtained. It achieves this by selecting sufficient features - from the original feature set - capable of representing the entire feature space and reconstructing them. Architecturally, it leverages the use of highly nonlinear latent representations in deep Autoencoders and intrinsically learns, in an unsupervised fashion, the relevant and globally representative subset of features through a customized Selective Layer. Extensive experimental results on three high-dimensional public datasets have shown promising feature selection performance by SDAE in comparison to other existing state-of-the-art unsupervised feature selection methods.

Outliers Robust Unsupervised Feature Selection for Structured Sparse Subspace

Outliers Robust Unsupervised Feature Selection for Structured Sparse Subspace

Unsupervised feature selection using chronological fitting with Shapley Additive explanation (SHAP) for industrial time-series anomaly detection

With the development of the industrial Internet of Things (IIOT), an amount of industrial multivariate time series (IMTS) data has been collected by various sensors. IMTS data anomaly detection plays an important role in industrial process monitoring and operation condition identification. Many real-world IMTS datasets usually have a large number of redundant features, which may lead to deviation of the anomaly detection model and misunderstanding of potential feature correlation. Hence, it is essential to select features that have critical and effective effects on anomalies and operation conditions. In this paper, a feature selection that introduces knowledge-based target variables into inner feature selectors (IFSs) is proposed as a wrapper method, referred to as ILSFS. The IFS is constructed by integrating a timing fitter, which is established based on the long and short-term memory (LSTM) network, with the Shapley Additive explanation (SHAP) assisted to eliminate unimportant features. The experiments were conducted on two datasets: the vertical roller mill dataset (VMD) of a slag milling factory in Jiangsu Province and the public server machine dataset (SMD). Compared with None feature selection, SVM-RFE, LGBM -Boruta, Genetic-based methods, and other SHAP-assisted feature selection methods such as XGBSVFIR and ShapHT+, the experiment results highlighted the proposed methodology ILSFS improves the best F1 score of each the above baselines by 0.2799, 0.2208, 0.2647, 0.1882, 0.3775, 0.2765 for the VMD and 0.1475, 0.0586, 0.0567, 0.0638, 0.0519, 0.2250 for the SMD on the downstream anomaly detection. The best precision of the ILSFS for the VMD and the SMD was 0.8224 and 0.9499, respectively, while its best recall was 0.7353 and 0.4902, respectively.

A Review of Machine Learning Algorithms for Biomedical Applications.

As the amount and complexity of biomedical data continue to increase, machine learning methods are becoming a popular tool in creating prediction models for the underlying biomedical processes. Although all machine learning methods aim to fit models to data, the methodologies used can vary greatly and may seem daunting at first. A comprehensive review of various machine learning algorithms per biomedical applications is presented. The key concepts of machine learning are supervised and unsupervised learning, feature selection, and evaluation metrics. Technical insights on the major machine learning methods such as decision trees, random forests, support vector machines, and k-nearest neighbors are analyzed. Next, the dimensionality reduction methods like principal component analysis and t-distributed stochastic neighbor embedding methods, and their applications in biomedical data analysis were reviewed. Moreover, in biomedical applications predominantly feedforward neural networks, convolutional neural networks, and recurrent neural networks are utilized. In addition, the identification of emerging directions in machine learning methodology will serve as a useful reference for individuals involved in biomedical research, clinical practice, and related professions who are interested in understanding and applying machine learning algorithms in their research or practice.

Exploratory Analysis of Distributional Data Using the Quantile Method

The quantile method transforms each complex object described by different histogram values to a common number of quantile vectors. This paper retraces the authors’ research, including a principal component analysis, unsupervised feature selection using hierarchical conceptual clustering, and lookup table regression model. The purpose is to show that this research is essentially based on the monotone property of quantile vectors and works cooperatively in the exploratory analysis of the given distributional data.

Application of Unsupervised Feature Selection in Cashmere and Wool Fiber Recognition

Application of Unsupervised Feature Selection in Cashmere and Wool Fiber Recognition

Unsupervised feature selection using orthogonal encoder-decoder factorization

Unsupervised feature selection (UFS) is a fundamental task in machine learning and data analysis, aimed at identifying a subset of non-redundant and relevant features from a high-dimensional dataset. Embedded methods seamlessly integrate feature selection into model training, resulting in more efficient and interpretable models. Current embedded UFS methods primarily rely on self-representation or pseudo-supervised feature selection approaches to address redundancy and irrelevant feature issues, respectively. Nevertheless, there is currently a lack of research showcasing the fusion of these two approaches. This paper proposes the Orthogonal Encoder-Decoder factorization for unsupervised Feature Selection (OEDFS) model, combining the strengths of self-representation and pseudo-supervised approaches. This method draws inspiration from the self-representation properties of autoencoder architectures and leverages encoder and decoder factorizations to simulate a pseudo-supervised feature selection approach. To further enhance the part-based characteristics of factorization, orthogonality constraints and local structure preservation restrictions are incorporated into the objective function. The optimization process is based on the multiplicative update rule, ensuring efficient convergence. To assess the effectiveness of the proposed method, comprehensive experiments are conducted on 14 datasets and compare the results with eight state-of-the-art methods. The experimental results demonstrate the superior performance of the proposed approach in terms of UFS efficiency.

Unsupervised Feature Selection With Flexible Optimal Graph.

In the unsupervised feature selection method based on spectral analysis, constructing a similarity matrix is a very important part. In existing methods, the linear low-dimensional projection used in the process of constructing the similarity matrix is too hard, it is very challenging to construct a reliable similarity matrix. To this end, we propose a method to construct a flexible optimal graph. Based on this, we propose an unsupervised feature selection method named unsupervised feature selection with flexible optimal graph and l2,1 -norm regularization (FOG-R). Unlike other methods that use linear projection to approximate the low-dimensional manifold of the original data when constructing a similarity matrix, FOG-R can learn a flexible optimal graph, and by combining flexible optimal graph learning and feature selection into a unified framework to get an adaptive similarity matrix. In addition, an iterative algorithm with a strict convergence proof is proposed to solve FOG-R. l2,1 -norm regularization will introduce an additional regularization parameter, which will cause parameter-tuning trouble. Therefore, we propose another unsupervised feature selection method, that is, unsupervised feature selection with a flexible optimal graph and l2,0 -norm constraint (FOG-C), which can avoid tuning additional parameters and obtain a more sparse projection matrix. Most critically, we propose an effective iterative algorithm that can solve FOG-C globally with strict convergence proof. Comparative experiments conducted on 12 public datasets show that FOG-R and FOG-C perform better than the other nine state-of-the-art unsupervised feature selection algorithms.

Joint Multi-View Unsupervised Feature Selection and Graph Learning

Joint Multi-View Unsupervised Feature Selection and Graph Learning

Unsupervised feature selection via dual space-based low redundancy scores and extended OLSDA

Spectral clustering is a widely used method for unsupervised feature selection (UFS) to generate pseudo labels. Nonetheless, it is acknowledged that graph algorithms suffer from issues such as redundancy and the dissatisfaction of connectivity, which greatly affect the quality of the learning local manifold. Moreover, existing UFS methods usually ignore the linear dependency among features and the role of non-negative semantics of the predicted labels. Hence, a novel algorithm using dual space-based low redundancy scores and extended orthogonal least square discriminant analysis (OLSDA), abbreviated as DLSEO, is proposed in this paper. Specifically, extended OLSDA is employed to derive non-negative clustering labels and a manifold structure, which avoids explicitly constructing a Laplacian graph. The dual space-based low redundancy scores related to mutual information eliminate redundant data and features, preventing their interference in the feature selection process. Moreover, ℓ2,1−2-norm is introduced to simultaneously guarantee the feature weight matrix's sparsity and ensure the selection of salient features. The results of experiments conducted on twelve benchmark datasets compared with several relevant methods show the superiority of DLSEO.

Soft Label Guided Unsupervised Discriminative Sparse Subspace Feature Selection

Soft Label Guided Unsupervised Discriminative Sparse Subspace Feature Selection

Multi-view unsupervised feature selection with consensus partition and diverse graph

Multi-view unsupervised feature selection has gained significant attention in effectively reducing the dimensionality of unlabeled data collected from multiple sources. Many existing methods integrate the tasks of graph learning and feature selection to select informative features. While these methods have demonstrated promising results, they usually construct a consensus graph by merging multiple graphs or consider the consistent graph of all views. However, due to the view heterogeneity, it becomes difficult to identify a shared similarity structure. On the other hand, the clustering outcome remains consistent across all views. In light of this, we propose generating multiple graphs that are as mutually exclusive as possible to enhance the complementarity between views. Additionally, our method bridges graph learning and consensus clustering to leverage the indicator consistency. We also present an effective algorithm to optimize the objective function. Finally, extensive experiments on six benchmark datasets demonstrate that our method outperforms state-of-the-art methods. The code is available at https://github.com/HdTgon/2023-INS-CDMvFS.

Robust autoencoder feature selector for unsupervised feature selection

Unsupervised Feature Selection (UFS) methods are known to produce models with excellent ability to select high-quality features. This advantage, however, is challenged when analyzing noisy real-world data, where anomalies are prevalent. For example, there may be a feature (anomalous feature) that is corrupted across many samples or a sample (anomalous sample) that has more corruptions than its peers. Previous literature focused on addressing anomalous samples, and methods that are robust to both types of anomalies have been under-explored. This paper proposes a novel general framework for reconstruction-based UFS methods, which can be embedded into the feature learning process to simultaneously remove anomalous samples and features. Specifically, the framework learns double binary weight vectors to assign 0 weights to samples or features with the highest reconstruction errors and 1 weights to the others when computing reconstruction errors. By discarding the 0-weighted samples and features when updating the model parameters, the anomalies in the data are excluded. This allows the model to focus more on learning from the clean part of the noisy data. Our proposed framework is then integrated with AutoEncoder Feature Selector (AEFS [10]) to develop a new method, which jointly performs anomaly removal and feature selection. The experimental results demonstrate the effectiveness of the proposed framework. Particularly, processing both types of anomalies provides better robustness than processing only one type. Moreover, our proposed method outperforms several state-of-the-art methods on various real-world datasets.